Action to auto publish PyPI #686

libialany · 2024-05-22T21:57:14Z

Fixes #669.

Thank you for reviewing my latest pull request, and I've made improvements in the following ways:

pypa/gh-action-pypi-publish@release/v1 accepts either a password or PyPI token. Utilize the token instead of the user's password also i change the variable name to pypi_token.
Added a secret variable for specifying the Python version. Referring to the documentation of actions/setup-python@v5 , I've set it to '3.11.9', aligning with the available Python versions for that GitHub Action.
Implemented the use of artifacts to transfer the 'dist' folder between jobs (actions/download-artifact@v4, actions/upload-artifact@v4).
Set condition for publishing, ensuring the last step succeeds, the secret is not empty, and the event is push and is a
tagged commit.

whimboo · 2024-05-28T11:37:00Z

@libialany thanks for the update. In case another update is required after my review please to keep working on this PR and please do not create a new one again. Thanks.

whimboo · 2024-05-28T11:39:05Z

.github/workflows/release.yml

+    name: Build package
+    runs-on: ubuntu-latest
+    env:
+      PYTHON_VERSION: ${{ secrets.PYTHON_VERSION }}


I don't think that we should make it configurable. Lets just hard-code to use 3.12. The same comment applies to the other placeholders below.

Thanks for your response, I will definitely do it.

whimboo · 2024-05-28T11:50:26Z

.github/workflows/release.yml

+          name: dist_directory
+          path: dist
+      - name: Publish package to PyPI 
+        if: steps.download.conclusion == 'success' && env.TOKEN != '' && github.event_name == 'push' && startsWith(github.ref, 'refs/tags')


Do we actually need the steps.download.conclusion == 'success' check here? If the download step fails we won't even reach the upload step, right?

Also without a token the job should fail with a clear message which doesn't seem to be the case right now. I would suggest to remove the TOKEN check.

Is there a way for dry testing those changes without publishing? Also I assume the tasks are triggered when a new release tag is being created? Maybe you could add a comment describing how it's run?

now I understand ghp-action-pypi-publish@release better and will proceed to fix the PyPi-publish job. Thank you.

whimboo · 2024-05-28T11:51:25Z

.github/workflows/release.yml

+        with:
+          name: dist_directory
+          path: dist
+      - name: Publish package to PyPI 


Suggested change

- name: Publish package to PyPI

- name: Upload release to PyPI

Thanks, I'll correct it.

whimboo · 2024-05-28T11:53:45Z

.github/workflows/release.yml

+      - name: Install dependencies
+        run: python -m pip install -U build
+
+      - name: Build package


Suggested change

- name: Build package

- name: Build a binary wheel and a source tarball

Thanks, I'll correct it

whimboo · 2024-05-28T11:54:04Z

.github/workflows/release.yml

+      - name: Build package
+        run: python -m build
+
+      - name: Upload dist folder


Suggested change

- name: Upload dist folder

- name: Store the distribution packages

Thanks, I'll correct it

whimboo · 2024-05-28T11:54:27Z

.github/workflows/release.yml

+      - name: Upload dist folder
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist_directory


Suggested change

name: dist_directory

name: python-package-distributions

Thanks, I'll correct it

whimboo · 2024-05-28T11:57:40Z

.github/workflows/release.yml

+    name: Upload release to PyPI
+    runs-on: ubuntu-latest
+    needs: build
+    env:


Based on the documentation we need the following?

environment: name: pypi url: https://pypi.org/p/<package-name> # Replace <package-name> with your PyPI project name permissions: id-token: write # IMPORTANT: mandatory for trusted publishing

My mistake, I will correct it

whimboo · 2024-05-28T12:29:32Z

.github/workflows/release.yml

+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          user: ${{ secrets.pypi_user }}
+          repository-url: https://test.pypi.org/legacy/


This is not our official package but points to a test instance of pypi.

Thank you. I specified a repository URL. There are two endpoints: TestPyPi and UploadPyPi. It's highly recommended to upload your package to [TestPyPi] (https://test.pypi.org/) first. I will fix my code.

libialany · 2024-05-30T16:47:26Z

I have corrected the errors I had. Could you please review it?, improved

whimboo

Thanks a lot for the update and sorry for the delay here. I would actually propose some more changes because I've enabled a Trusted Publisher for this PyPI project. Please check my inline comments for more details.

.github/workflows/release.yml

libialany · 2024-06-12T18:39:24Z

I have corrected the errors I had.improved version. Thank you very much for the link. I didn't apply a test job because I believe the tests are fully covered with test.yml. Could you please review it?,

whimboo

Thank you for the update. Scanning over the code it looks fine to me, and I think that we should get this PR merged. I expect some quirks to be still around with the trusted publisher, but those can be fixed when I'm going to try to release.

Nevertheless when I was triggering the workflow for tests on this PR all jobs failed. It's not related to your changes, but test files not being able to find on the remote server. I have to do a separate PR for that. But I would suggest that you add a test job to this workflow as requested earlier so that we could identify last minute failures and do not have to follow-up with a bugfix release shortly after. Thanks!

whimboo · 2024-06-26T09:44:57Z

FYI I created a PR to fix the tests at #692.

… failures.

libialany · 2024-07-02T16:42:29Z

Thank you!. I have improved the release.yml and attempted to integrate the test.yml workflow:

workflow_run:
  workflows: [ Test ] # Name of the test.yml workflow
  types: [ completed ]    # Trigger when the test workflow completes
  branches: [ master ]    # Run in the master branch; the branch must be the default branch (if you specify a branch that isn't the default, it will never run)

I also added a condition to the deploy job to check if the completed test workflow was successful:

if: ${{ github.event.workflow_run.conclusion == 'success' }}

I will learn more about testing in GitHub Actions and will review the documentation at #692.
link resources:
branches
work_on

.github/workflows/release.yml

whimboo

Thank you for the latest update! I think that we are now at a stage where we basically should land all the changes related to the release workflow. When the next release is ready I'll try it out and if it fails will provide a follow-up patch.

Thank you @libialany for all the hard work to make this workflow a reality! It will clearly help to better automate the release process of mozdownload, and maybe we could even add more in the future like bumping the version number, updating the changelog etc...

libialany added 5 commits May 19, 2024 19:31

feat: GitHub Action to auto-publish to PyPI

25bc22c

feat: build isolation feature of pip

a884302

feat: updated GitHub Action versions

18d8275

fix: Refactor jobs, streamline runs-on, and optimize conditionals.

c3b3ea3

fix: removed unnecessary file

a812386

whimboo mentioned this pull request May 28, 2024

Create GitHub Action to auto-publish to PyPI #683

Closed

whimboo requested changes May 28, 2024

View reviewed changes

fix: improve jobs, change messages

1dd27ee

whimboo requested changes Jun 12, 2024

View reviewed changes

.github/workflows/release.yml Outdated Show resolved Hide resolved

.github/workflows/release.yml Outdated Show resolved Hide resolved

.github/workflows/release.yml Outdated Show resolved Hide resolved

.github/workflows/release.yml Outdated Show resolved Hide resolved

libialany and others added 2 commits June 12, 2024 14:00

Merge branch 'mozilla:master' into auto-publish_PyPI

111f358

fix: fixed job deploy

292442c

whimboo requested changes Jun 26, 2024

View reviewed changes

libialany and others added 2 commits July 2, 2024 12:05

Merge branch 'mozilla:master' into auto-publish_PyPI

879995e

feat: added release workflow to trigger on test completion and handle…

803d60f

… failures.

whimboo self-requested a review July 4, 2024 08:30

whimboo reviewed Jul 4, 2024

View reviewed changes

.github/workflows/release.yml Outdated Show resolved Hide resolved

libialany added 2 commits July 5, 2024 08:08

Update release.yml

f554d00

Update release.yml

ba7e599

whimboo approved these changes Jul 17, 2024

View reviewed changes

whimboo merged commit 1959681 into mozilla:master Jul 17, 2024
16 checks passed

whimboo mentioned this pull request Jul 17, 2024

Create GitHub Action to auto-publish to PyPI #669

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Action to auto publish PyPI #686

Action to auto publish PyPI #686

libialany commented May 22, 2024 •

edited by whimboo

Loading

whimboo commented May 28, 2024

whimboo May 28, 2024

libialany May 28, 2024

whimboo May 28, 2024

libialany May 30, 2024

whimboo May 28, 2024

libialany May 30, 2024

whimboo May 28, 2024

libialany May 30, 2024

whimboo May 28, 2024

libialany May 30, 2024

whimboo May 28, 2024

libialany May 30, 2024

whimboo May 28, 2024

libialany May 30, 2024

whimboo May 28, 2024

libialany May 30, 2024 •

edited

Loading

libialany commented May 30, 2024 •

edited

Loading

whimboo left a comment

libialany commented Jun 12, 2024

whimboo left a comment

whimboo commented Jun 26, 2024

libialany commented Jul 2, 2024

whimboo left a comment

	- name: Publish package to PyPI
	- name: Upload release to PyPI

	- name: Build package
	- name: Build a binary wheel and a source tarball

	- name: Upload dist folder
	- name: Store the distribution packages

Action to auto publish PyPI #686

Action to auto publish PyPI #686

Conversation

libialany commented May 22, 2024 • edited by whimboo Loading

whimboo commented May 28, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

libialany May 30, 2024 • edited Loading

Choose a reason for hiding this comment

libialany commented May 30, 2024 • edited Loading

whimboo left a comment

Choose a reason for hiding this comment

libialany commented Jun 12, 2024

whimboo left a comment

Choose a reason for hiding this comment

whimboo commented Jun 26, 2024

libialany commented Jul 2, 2024

whimboo left a comment

Choose a reason for hiding this comment

libialany commented May 22, 2024 •

edited by whimboo

Loading

libialany May 30, 2024 •

edited

Loading

libialany commented May 30, 2024 •

edited

Loading